Sparse Approximation with Block Incoherent Dictionaries

Building good sparse approximations of functions is one of the major themes in approximation theory. When applied to signals, images or any kind of data, it allows to deal with basic building blocks that essentially synthesize all the information at hand. It is known since the early successes of wavelet analysis that sparse expansions very often result in efficient algorithms for characterizing signals in noise or even for analyzing and compressing signals. The very strong links between approximation theory and computational harmonic analysis on one hand and data processing on the other hand, resulted in fruitful crossfertilizations over the last decade, from fundamental results (near optimal rate of non-linear approximations for wavelets and other basis [1]) to practical ones (like the JPEG2000 image compression standard). Natural signals however do not generally lend themselves to simple models, for which orthonormal basis are generally near optimal. Images for example do contain smooth parts and regular contours that could be efficiently represented by a curvelet tight frame [2], but they also contain various kind of irregular edges together with a plethora of textures. Audio signals contain sharp transients and smooth parts that are suitable for wavelet basis, but they also contain stationary oscillatory parts that are better suited for local trigonometric basis [3]. Bearing in mind the multiple components of natural data, one is tempted to approximate them with mixtures of basis functions. Approximating data with general dictionaries seemed a daunting task, and raised many questions concerning the unicity and optimality of sparse representations. Fortunately there have been recently an intense activity in this field, showing that constructive results can be obtained on all fronts. The possibility of recovering optimal sparse representations using Basis Pursuit (BP) opened the way [4]–[7]. When an exact sparse representation is not needed, approximation results become more useful, and recent results have shown that variations around greedy algorithms such as Matching Pursuit (MP) and Orthogonal Matching Pursuit (OMP) are promising [8], [9]. One of the key properties in the above-mentioned results lies in the characteristics of the dictionary, and one could roughly say that in most cases the latter is required to be sufficiently incoherent, i.e close enough to an orthogonal basis. Putting strong restrictions on the dictionary though may damage the original goal in the sense that we loose flexibility in designing it. In this paper, we basically relax some of these strong hypotheses by allowing more redundancy in the dictionaries, through the concept of block incoherence, which basically describes a dictionary that can be represented as the union of incoherent blocks. We show that even pure greedy algorithms can strongly benefit from such design by proving a recovery condition under which Matching Pursuit will always pick up correct atoms during the signal expansion. Based on this result, we design an algorithm that constructs a near block incoherent dictionary starting from any initial dictionary. A tree structured greedy algorithm is then proposed as a way of constructing sparse approximations with block incoherent dictionaries. This algorithm presents the important advantage of being much faster than a classical Matching Pursuit. In the same time, it only minimally degrades the quality of approximation thanks to the recovery condition, derived for block incoherent dictionaries. The performance of the proposed algorithm are demonstrated in the context of image representation. This paper is organized as follows. Sec. II first proposes definitions on coherence between generic subsets of basis functions. It then proposes theorems which show that Matching Pursuit picks the correct atoms during signal expansion, provided that the dictionary is block incoherent. Sec. III presents a generic method to build block incoherent dictionaries from any set of functions. It finally shows the benefits of the recovery condition in the context of image representation using block incoherent dictionary.